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Tltad of measuring dimensional changes in an object milisrng that memod and 
an aircraft structure. 

There is otten ^placements of large structures such as 

is undergoing a mechamcal test, or enually the p 

thatexcesslvedisplacementswrtoanobjecttotarehkdyto 

v. «^*,*nieal tests is termed extensometry and has 

wed Tins is not practical for some fragile or small structures. Addmonany, 
environments having chemically corrosive atmospheres. 



- are known that utilise captured video imagesfor the measurement of 

commencement of monitoring. 4. „ f „ H ect that can be monitored 

or targets, to the objeet under test limits me rmmmum size of object that can 
using such prior art video monitoring systems. 

v „ •* 4, beneficial to monitor the stresses and strains of a structure is the 
^r^.ohaveaapecificn.under.e various ioadings 
aerospace mdustry. Ancraft are gn ^ 

======== = 

*. ahftame has occurred leading to loss of Ufe and great econormc costs The opera 
^on an as-requueu basis rafirer than toafixed schedule. 

monitoten so that fire condition of *e -J^Z* However, monitoring the 

^^ZZZ^Z^ of the ahcrafi, — is »ore 
airframe's response to flight loads anO ^ 

— . -mr^^ - — — : 



„ other sensors Whilst these sensors are very effective porting local 
—7 ** °!I iTsful a, «- ^ of to aircraft's stiucftraf 

conditions fhey are less successtul mbstaItia i investment in distributing 

resp0 nse. W.tonset.eseappoachesre^asnb^-v ^ 



aircraft. 



rt of the present invention there is provided a method of 

toto estoate of the posfton of fte treated f ^ 
with the further estimate of the position to derive renn 

Prefer the tirst and second determining steps con^e — g — ^ 
- hnage using a pred— — „ „ . 
feature vritbiu the image to the nearest ptsel md fitttog to 

^ of pixel ^^^1*..** 
Pl^o^toa^erf^onandd^^--^ 

its turning poi* whereby to posmon correspond^ to ft 
^stopositionoftofeati^to^to^tionoftopnceh 

^ • 1 C ^ proximity to the determined position 
Additionally, to sub-pixel positions may be m closer proxnnrty 

than the pixel positions. 

method may totor comprise deterfthting to "^^Z^ 
determining the difference in position of the featur 



w 4 

• 'j image such that said feature is substantially in 

- Xbyanincre.ed resolntio »»— — ofsaidfeaturersproduced. . 

A « or — y> *e predated — «— - — 3 
"normalised greyscale correlation function. 



least once. 



. nf the present invention there is provided a meflrod of 
According to a seeond aspect o£ the . ^ ^ ^ 

m « dimensional changes » an object, the > tQ me 

object to determine any dimensional changes. 

• , , „f the nresent invention there is provided positioning apparatus 
*^«^XJZ£Z appa^compri^animagecapb^deviea 

present invention. 

rilaye.Lgeandi^theselectedo.eettotbeimageproeesso, 

Accord, . a - aspect - *e P^ — ^ ^ 
de^g dimensional changes » an ob^ * = appar P ^ ^ 
aevice axrangedto sequermaUy provide aplurahty of nnag* * enc J * 
^ecomp^aWofpixels.anirnageprocessorarrangedtose.u 



togeand determine if the determined position changes between images. 

,• to a fifth aspect of the present invention there is provided a method of 

=2= ~ — =r 

me thod of me first aspect of the present invenbon, and caleulahng the 

m ttS of thefeatme given the determined feature coordmates m each rmage and the 

determined relative positions of each image capture device. 

According to a sixth aspect of the present invention mere is provided a mefbod of 

w— — *~ * - object - te me * od 51 1 1 

■ time fiom one anomer; determining the 2— ional posrtron of me objec 

JHZZLm H accordance with the method of the first aspect of the presen 
« luting me 3-dimensional position of me obiect from me first and second 
11 ttmmlsionai position and spatia, reiationsbip of the image captirre 

According to a seventh aspect of me present invention there is aiso provided a metirod^f 
^coordinatesofafeaturewitrmranimagetevdthmafi^onofap.tel.the 



* 



determine its turning point, whereby to coo— corraspontog to to «*, poto 
pevidetocoordinatesoftofeatae to within .aid taction of a prxel. 

By evaluating to correlation function at sub-pixel positions, tot is positions having 
III Uprising fractions o f a pixel (to example (15V, 3*,,), to 

estimate of the feature position. 

According toatotor aspect of to present invention tore is also 

Z* «i» devices to a tot portion of an. aircraft struck, attantog one or mo 

Ige caphrre device and processing to captured images to detemnne to magnrtude 
and/or direction of movement of to or each target 

Metrology technics based on to acquisition and manipulation of video data streams are 
Z degree of accuracy. From to data on to displacements, to deformatmn of to 

^Lgs can men be summed over toe, for example, to indicate to rate at wmch 
— A^onaUy.comparisonofto^— 

til structures can provide information on to shuctural condino a oTto . 
For example, monitoring wing and fuselage deformations provrdes mfo— on to 
filelage-s response to to wing loading. Changes in tos response over to 1* of to 
^ 1 Jcoupled to structure, response and -wear-ouf models to provrde mdrcaUon 
71 rUoation conations tuning investigation or to residual togue hfe 
totoahtame. ^y.toaflyby wi« aircraft to hrforma«on on to relate 
of different par. of to ahcraft- a « can be coupled m. to a, craft 
STL system so as to provide feedback control to minimise aircraft manoeuvre 



loads. 



Preferably the method comprises attaching at least three targets to the second portion of 
the aircraft structure at known relative positions to the second portion of the aircraft the ax 
least three targets being within the field of view of the or each image capfirre devrce, 
hereby the position and orientation of said second portion of the aircraft relative to the or 
each image capture device is determined by said processing step. 

Additionally, at least one finther target may be provided mounted orthogonally offset ftom 
& e plane of the other targets within the same field of view of an image capture devrce. 
This has the advantage of magnifying the movement of the firrther target due to bendmg of 
the aircraft structure to which it is attached. 

Alternatively, a reflective surface, such as a mirror, may be attached to the aircraft 
structure such that at leas, one of the targets is within tire field of view of an image 
capture device when viewed through the mirror. 

Each image capture device may be arranged to automatically identify the tatgets as items of 
interest. This allows a target to be automatically reacquired should it be momentary 
obscured from the view of the image capture device. 

According to another aspoct of tire present invention there is provided arr aircraft structure 

to a first portion of an aircraft structure; at least one target arranged to be attached to a 
second portion of the aircraft structure, whereby the at least one target is within the field of 
view of the at least one image capture device; and an image processor arranged to recerve a 
ptafity of images from the at least one image capture device and procass said recerveti 
images to determine the multitude and/or direction of movement of the at least one target 

Additionally, the target may be in the form of an iUuminated panel that includes ateas of 
differential Rumination The fflorninafion may be provided by LED's or other similarly 
reliable light sources. Such targets minimise the influence of ambient fight levels on the 
accuracy of the monitoring apparatus. 



w 

Additionally or alternatively, a flexible tight guide may interconnect tire image capture 
device and target Ttie- tight guide may be a sealed, flexible babe or other stable-means 
provided to act as a shroud to prevent people and/or- other objects from obscunng the 
targets from the camera and to reduce or eliminate the effects of changes in ambtent 
lighting. 

Additionally, the image processor may execute arty one of tire methods of the aspects of the 
present invention described hereinabove, although other positional determining techmques 
may alternatively be applied. 

Embodiments of the present invention are described below, by way of illustrative example 
only, with reference to the accompanying figures, of which: 

Figures la to lc illustrate the choice of neighbouring pixels to the position of the best 
matched pixel for 4, 8 and 24 neighbouring pixels respectively; 

Figures 2a and 2b illustrate the effects of aliasing between a reference image and 
comparison image; 

Figure 3 schematically illustrates the positional errors typical of normalised correlation; 

Figure 4 schematically illustrates the projection of a pixel array onto a fictionally shifted 
pixel array in accordance with an embodiment of the present invention; 

Figures 5a to 5c schematically illustrates me sub-pixel positions of neighbouring sub-pixel 
points about the position of a best matched pixel; 

Figure 6 schematically illustrates video metrology, apparatus according to an embodiment 
of the present invention; 

Figure 7 schematically illustrates an aircraft structure monitoring apparatus according to an 
embodiment of 1he present invention; 



the 



Figures 9 a and » schematic^ «— * - " ~ ^ ^ ~ 
beading on a structure; 

^•^ ~f „ n offset target according to an 

Figure 10 schematically illustrates the add*on of an offset 

embodiment of the present invention; and . 

„ orrW n f +he aircraft monitoring 

Figu re 11 schematic* H» * 
apparatus of Figure 7. 

• method of deterrnining the (X,Y) 

^ io — «j — °;jz:^L^^^-« 

pcsition o f a — . - device) ^ and to a 

captured using a conventional CCD (charge c p on an image on a 

a. stream that a computer can manipulate „ a £M*£ ^ ^ 

connected computer mom., hounding regroup - ^ fc 
^ that the user ^es to momtor *e posmon of 8 ^ ^ 

pattern associated mose readies and <^«»£ ^ idmtjfication of the 

Icuontomeasure thedegreeof 
Tetnplate.matctangrs performed by « Stoted correlation Sanctions used 

* e suntiarity hetweentwo images, or pa* «^ ^ Euc hdean 

.elude Kormatise d ^ ^rTy — " 

rr:r;r:u^r^^---^ 
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captured by the bounding region is at a position (X, Y) in the image. The correlation 
function is evaluated at a number of (X, Y) positions in the image to determine the position 
that gives the best value of the correlation function. This yields the (X, Y) position of the 
feature within the image to the nearest pixel. 

Having located the position of the feature to the nearest pixel, its sub-pixel position can be 
estimated. The correlation function is evaluated atN surrounding positions. Figures la to 
lc schematically represent a pixel with 4, 8 and 24 nearest neighbours respectively. In 
each of Figures la to' lc each small square 2 represents an individual pixel, with the 
central, shaded square 4 representing the. position of me feature of interest, to the nearest 
pixel. The neighbouring pixels at which the correlation function is evaluated, are indicated 
by the squares having black dots in them 6 . The values of the correlation function at the N 
surrounding pixels are fitted to a function, such as a bi-quadratic, typically using a 
least-squares fitting technique. The sub-pixel position of the feature may then be estimated 
by differentiating the quadratic and calculating fee position of its turning point since the 
maximum fit will occur at a turning point in the differentiated quadratic. The number N of 
surrounding pixels is chosen to ensure that N is at least equal or greater man the number of 
free parameters in the selected function, so that the function can be solved. 

A typical accuracy for a good implementation of template matching using normalised 
greyscale correlation is in the region of 0.01 pixels, under ideal conditions. However, for 
certain applications such as those proposed in the monitoring of mechanical testing, or the 
movements of large structures, this accuracy is inadequate. Furthermore, the NGC 
approach has an additional technical limitation as is explained with reference to Figures 2a 
and 2b. In each of Figures 2a and 2b, each individual square represents a single pixel, with 
the reference feature of an image represented by the filled boxes 8. As is well known, each 
pixel may have a specific greyscale value anywhere between, and including, black and 
white. In the example shown in Figure 2a, the feature is composed initially of a series of 
pixels of the same greyscale, either black or white. If the next captured, or comparison, 
frame of the video image represents a shift of exactly an integer number of pixels, tnen the 
pattern of the comparison and reference images will be an exact match. However, if the 
comparison image has moved a non-integer number of pixels then the pattern will be as 



seen inFigure 2b. The pattern isnowbounded by pixels 10 in which tire greyscale value is 
between black and white as represented by the cross-hatehing, and the pattern is clearly 
visually distinct ftomlbat of the reference feature. This effect is known as aliasrng. 

Tte difference in pattern due to the movement across pixel boundaries of the reference 
feature of interest does not disrupt the ability of the NGC approach described above to 
monitor the (X, Y) position toalevel of accuracy in the region ofO.01 pixels. However, rf 
4. reference feature of interest is displaced by a known amount for each frame over a 
S eries of frames, the level and type of error between the actual ancPestimated (X, Y) 
position can be found. When this procedure is carried out it is seen that.the normahsed 
correlation approach generates bothrandom and strucbrred, or systematic, errors due to the 
aliasing effects. This is represented schematically in Figure 3. The stratghtlme 12 
represents the true X or Y position of the reference feature of interest, whereas tire markers 
14 show tire data points that might be calculated using tire NGC approach. It can be seen 
tot the discrepancy between actual and estimated positions have a random error, 
represented by the deviations of each data point ftom tire general curve defined by them 
and a systematic error represented by tire deviation of the general curve ftom the sttarght 
linen.thisbeingtypicaloftheresultsoftheNGCapproach. 

Embodiments of the present invention utilise the NGC approach to gain an initial estimate 
of the position of the reference feature of interest. However, two additional operations are 
performed to both eliminate tire systematic error and refine the best estimate of the posrtron 
of the reference feature of interest to give the maximum resolution. 

to embodiments of the present invention, the standard NGC technique is used to calculate 

projecting the pixel array onto a new pixel array that has been slnfted by a ftactional part of 
' a pixel, such that the sum of the pixel translation and the fractional par. of the estimated 
sub-pixel position is an integer value. That is to say if tire position estimate is <*■/„ ? U) 
the pixel array may be shifted by -V, a pixel in the X direction and -% a pixel in the Y 

*» a pixel in the Y direction. Usually the choice of direction of translation would be such 



50 33 t0 minimise the magnitude of the — It ^ afco be appreciated ft* the 
L^emayaitern^velyUtmn^ TM.hflW—W*- 

^LoeensBfteaby^^onalp^of^o^^pixelpos^— 
^onal par. of me sub-pixel position estimate (e.g. half a pixel) is represented by DX, 
ST*L in Figure 4. The sampled pixe.s will now loo, more like the reference 
ZZ ! the resampling process approximately models tbe aUasing effects Unstinted m 

L a greyscale value cioser to blank or white or where the greyscale value was 
2c or wL it remains so. The correlation function is then evaluated at the be, 
^bedpositionanditsKsurroundingneighbours.d.e ^^T^ 
JstandardKGCtechni^. The fractional part (DX, DY) of the original sub-pixel 
estimate is added to die re-evaluated best matched position to give a refined 
^.estimate. This procedme may be lte raten a number of times to fmther refine the 



Afat ber improvement hi accuracy is achieved by ^g n,e N neighbourn^ saniples o 
Loirelationfirnctionat sub-pixel^ 

X may be derived and me correlation faction may be eva,uated at .y re, 

S the vail of me oorreladon Action evaluated at me N surroondmg pos.0. . 
fitted may better approximate me true correlation surface near its peak value if me N 

fcoNneighborus are chosen to be .close to the positionofthe best matched pixel. 
Additionally or alternatively, to finther refine the best estimate of tho position of the 
^wnto me skilled -«^« — 

Lormoresimfiarimages. The plurahty of images are aligned in complete re^i^o 



iJ 

, • + „ n^we a sinele, high resolution image, (i.e. 
any known Super Resolution technique, to produce single, gn 

more pixels than any of the originals). 

^ jZlitZa sue. that the feature appears in 

^^^10™ theta^^ set. A high resolution version of the feature can then be calculated. 

accomplished to a higher accuracy. 

Kesolu.cn technic as g enerall y the monitoreo 
each^eUonlvn^ginaUv^erentnonrauhse^^^ 

. auows the use of simple low pixel connt cameras to approach the 
possible with much more costly high pixel count cameras. 

strain gauges or LVDTs. 



to the test sample 22- or 32 . pressor also 

t^et location 24 and proves vtdeo data 30 * » to nt 

1 i™H data from the test machine. The processor <ur & 
receives sample load data from wiliabo ve A monitor 34 is coupled to 

processing performed by the processor. 

or ^ from a seance ^^^^ „ ^ „ f the test 
positions allow any dimensronal chafes, such * ' fimagesrece ived 

— ^rlLdeiv seated in 

fom the video camera 28 may he each vrdeo ftame, or may 

time as is appropriate. 

. . included wiihin the equipment to allow manual 

or automatic selection of an object 

sector may, for example, comprise circnihy «- * J— ^ ^ 

sup^sedontheh^edisplayedon^m^^ ^ ^ ^ 

^Cepl----^-^ 0f *^ tote ^ 

^ .deo caphrre and arraiysis apparatns may ^y 

«Wprt heins detenninable to a high level 01 awua , 



^^ve.^rftobe^ted.p^^— to be ta,en of fte 
scale msensMvity mherent in video metrology. 

— rrrrr™ 

t M il I. as microchips in a product setting, ft. measurement of 

to be able to obtain precision movement data m toree 
potential fields of application. 

«-«^— r^rtrtnerasl^a^toa^ 

Po^^orpo^onfteanc^^^ structure, 
of targets 46 some distance away, also ngioiy a 

as, but not exclusively, grey scale norm 



or twisting, of the fuselage. 

46 and 40 remain wwwm the two targets changes. 

line s50, 5 2a I e n olongerpar a Uel ; soti 1 ed 1 stanoebetwe m &etwo 

^lyfh^fo^on^^xeadaymea^lerange. 

^ts * — . * —~ -V P— 
^channeU^ofltaittogob^oaissuea. b . . 

offcetotafl* of the deflections ta the -ta. as areas of 

- — 15 - " — - - — 



h is to use a sealed monitoring unit, as also shown in Figure 11. 

» to**— ^ V * toexample.betwe.nfiaselagetenesinfteeaxgo 

«— -^-T- 1 ^^alTa^ud^tpeoplea^orooiects 

r:="-— ----- -™- 

lighting. 

— - * — - ^r^r;I— herem is oapable o f a 
operas. The high accuracy vrdeo gaugmg PP n4aoimB<ta- |. low cost 

^era will have animage field of more than 500 pnrels, gr 



100,000. 



resp onse can be calculated. K the unase field £ J fc detected .e of the 
phy^^oflOO^s^^^^^^^^d 
order of 1pm. If fire inifial drstarrce between » g ^ 
me angular deformation in the torsronal of 0.1°, due 
i * n nn?° For a change in angle trom mc v^u. r 
targets is of me order of0^ For ^g q ^ 50ffim ^ is less tfian 

- bending, fire apparent ^md, ^ ^ set , momt ed as sbown m 

0 , lm , which is not resolvable. However, dtoensional 

- » ->■•■ *• nrss^i — . 



m be used to shine on a translucent screen, with that laser 
s „asmahLBD laser — 
^oein^enastneW ^ 
difficulties due to the response 

offset. ■ _ 

- • «-f rt^ tareet can be measured in 
M1T . pras the position of trie target wu 

3D. A single camera is suioi 

that has 3 or more targets attached.. 

~ - — — ^rc^ - 

md !ens systems determmed. Wmg ^ te 

lections. However, Ms would oot ^ ^ ^e fieid end 

„ of a difiereot leos system to prov.de - ««P J by _ appiopriate 

0D ^ ^r^eand^ct^Vsafe^teoanceor 
-^^T^^r-ve.severeever.^Mo^^ 



CLAIMS. 

A M of — - coo— 0f . _ ^ ^ _ *e una, 
1 r-n, Pixels me method comprising: 

comprise: . ^determined correlation function 

todetermmethecoordmatesofthefeaturew. ^ ^ 

evalua tin g ttie — — ^J^lUlo.v^.a 
nei ghbourhood of the determined coordrnates and fitting 

tottet function; and ^ wmine to turning point, vteeby the 

within asaid fraction of aprxel. 
plurahty of sub-pixel positions. 
' toftedeterrnmedooordinatesti.anthepixelpositions. 
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image; 



^a^dresolutionreprese^oaof.aidfeatoeisproduoei 

A me thod according * any one of — * - 4, * 

r , ™ receding claim, wherein the translating step, second 
A method according to any preceding ckuui, 
determining step and summing step are repeated at least once. 

m t:r B ii o r:Zii *- • — - — 



* *r determining the position of an object, the apparatus 
9. Positioning apparatus for deteranmng m p 

comprising: encompassing the object, the 

an image capture device arranged to provide an imag 

„• . .1,™ 9 further comprising a monitor arranged to 
^ve^ddisplaythe^edmrageando^ ^ ^ ^ 

object within the displayed image and identify the 



processor. 



an image capture device arranged to sequeu 



*■ m claim 11 ««*-' fflranBed ,0 dBtSrmilie *" 

• • . *. 3-dimensional coo— of a feature, the method 
13. A method of determmmg the 3 dimens 

comprising-. h to capture an image 

providing at least two .mage capture devrc 
toclndingthefeature; . of said image capture devices with respect to one 

determining the position of each of sard .mag 
fflotber ' f^fearurev^eachimageinaccordancevrithtire 
determhtingme — ^^7— g the 3-dhnenaionai — of 
method of any one of clanns 1 1» 7, ^ ^ ^^cd 

Z feature given tire determined feature coordmates 
depositions of each image capturedev.ee. 

providing at least two image capture dev.ces 

another, frtwoHec t with each image capture device, to two 

capturing at least two images of me ohjectw. 

determining tire 2-dimensional posruon of ft. 
.ccordanoevrithdremethodofanyoneof clanns^ o ^ ^ ^ ^ 

coating the 3-dimensionar of fbe image capture 

^ ftom the 2-dimensional posmon and spa 

devices; and .„ ^eterarme any dimensional changes. 

co m parmgme3-toensionalpos.tionstodetermme 



,0 claim 9, ft*, comprising at least one ftrmer -age 
»■ ^onship to me first image capture device, and 

^e device ^ ^^^ to ^ me position of me object 

:r=r" - — — teKby 

3-dimensional position ofthe object. 

fc u OT u farther comprising at least one 
16 . Measurement apparatus «~*^ rf- ^ lUp t0 fce fimt image capture 
fete image cnpture devrc* at * P a todetemijl e me position of 

:r.r= - - - — * ereby — 8 te 

3 .dimenaional poaition of the object. 

W *e coordinates of a feature within an image to within a 
17. A method of determmmg the coor ^ me4od eompnsmg: 

former function; and maximum, whereby the 

„tiating the ftrihet ftnction to d-~- * fc ^ fc 

grates corresponding to the maximum provrde tire 
within aaid traction of a pixel. 

• ft *ucture the method comprising the ateps of 
18 .Ame<bod of monitoring an ^^^^^ofanairoraft structure, 
attaching one or more image captures devmes to ^ _ ^ 

-^-"^Tr^lrrinnagecapmre device, capturinga 
^get being wimin the field of vtew ^ ^ ^4 

seriea of images fiom the or each ^ J^^J,^ or eachtarget 
^to deterrnmememagnimde and/or drrecnon of 

, • ,8 wherein said method comprises attaching at least 
19. A method according to ^ at too wn relative positions to 

three targets to the second portion of the an 



by said processing step. 

♦ l.™ 18 or 19 wherein the or at least one of the targets is 

is attached. 

* ♦ i«™ 18 or 19 wherein a reflective surface is attached to the 

„f .kims 18 to 22, wherein the processing step 

23 A method according to any one of claims 18 ^ 

' comprises the method of any one of claims 1 to 8, 13, 14 or 17. 

r i„: mc 1 r tn 23 wherein the aircraft structure 

24 A method according to any one of claims 18 to 23, wn 
comprises aportion of a wing,.empenna g e or fuselage. 

aad/or direction of movement of the at least one target. 
plm e in .hichlhe at least one target is arranged*, be attached. 
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position to ™tbin . ftaction of a p«el and -™ of to 

estimate witia tiae furtiter estimate of tiae posnaon to arnve 



feature position. 



(Figure 4) 



